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Block Iterative Reweighted Algorithms for 
Super-Resolution of Spectrally Sparse Signals 

Myung Cho, Kumar Vijay Mishra, Jian-Feng Cai, and Weiyu Xu 


Abstract 

We propose novel algorithms that enhance the performance of recovering unknown continuous¬ 
valued frequencies from undersampled signals. Our iterative reweighted frequency recovery algorithms 
employ the support knowledge gained from earlier steps of our algorithms as block prior information 
to enhance frequency recovery. Our methods improve the performance of the atomic norm minimiza¬ 
tion which is a useful heuristic in recovering continuous-valued frequency contents. Numerical results 
demonstrate that our block iterative reweighted methods provide both better recovery performance and 
faster speed than other known methods. 


Index Terms 
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I. Introduction 

Compressed sensing promises to perform signal recovery using a smaller number of samples 
than required by the Nyquist-Shannon sampling theorem. In the compressed sensing framework, 
a sparse signal x is recovered from the observation vector y even though the dimension of y is 
much smaller than the dimension of x. Since compressed sensing reduces the sampling rate in 
recovering sparse signals, it has made great impacts in various signal processing areas 0]|. 

Compressed sensing has also found application to the problem of line spectral estimation, 
which aims to estimate spectral information from few observations. Early-stage compressed 
sensing frameworks for spectral estimation j2l El assumed that the frequencies of spectrally 
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sparse signals were located on discretized grid points in the frequency domain. However, in 
practice, frequencies can take values in a continuous domain, giving rise to the so-called basis 
mismatch problem |3] when the discretization of the frequency domain is not fine enough. 

The breakthrough theory of super-resolution 0 proposed by Candes and Femandez-Granda 
states that sparse continuous-valued frequencies can be exactly recovered through total variation 
minimization using a set of n uniformly spaced time samples, provided the minimum separation 
between any two frequencies is 4/n. In order to recover continuous-valued frequencies from 
few randomly chosen nonuniformly-spaced time samples, Tang et al. proposed off-the-grid 
compressed sensing that employs atomic norm minimization for frequency recovery 0. Later, 
it was shown that the t\ minimization over the fine discrete dictionary provides an approximate 
solution to the atomic norm minimization [0. 

In this paper, we are interested in recovering spectrally sparse signals with as few random 
time samples as possible. It is then natural to ask whether there are efficient frequency re¬ 
covery algorithms that can further improve the performance or relax the frequency separation 
conditions when compared with the total variation minimization or atomic norm minimization. 
We propose new iterative algorithms to enhance the performance of recovering continuous¬ 
valued frequency. In our iterative algorithms, we estimate the frequency support information from 
previous iterations, and use the support information as block prior [j71 for reweighted atomic 
norm minimization in later iterations. Numerical results show that we can improve recovery 
performance by exploiting the block prior provided in earlier iterations. 

We remark that there are quite a few works in the literature [8-13 J where iterative reweighted 
methods have been used to improve sparse recovery performance in compressed sensing. How¬ 
ever, the sparse signal recovery is considered over a finite discrete dictionary in [84121. Besides 
lfT3l . only our work considers recovering continuous-valued frequencies by directly reweighting 
in the continuous dictionary through a semi-definite program (SDP). Our work differs from 
m in that we provide different reweighting schemes that lead to improved signal recovery 
performance. In lfT3ll . the authors set the reweighting weight w(f) for a frequency / e [0,1] 
according to correlations between frequency atoms (see e.g. Theorem 3 of |[T3l ). In contrast, 
our method allows w(f) to take more general forms through the dual program of weighted 
atomic minimization under general weights llT4l . thereby lending more flexibility to incorporating 
external prior information and prior information passed on from earlier algorithm iterations. 
Numerical experiments show that our iterative algorithms improve both the recovery performance 


3 


and the execution time, compared with [5]| and lfT3ll . 


II. Background on Standard and Weighted Atomic Norm Minimization 

Algorithms 


In this paper, we denote the set of complex numbers, real numbers, positive integers and 
natural numbers including 0 as C, R, Z + , and N respectively. We reserve calligraphic uppercase 
letters for index sets. When we use an index set K, as the subscript of a vector x or a matrix 
F, i.e., xjc or F/c, it represents the part of the vector x over index set K, or the columns of the 
matrix F over index set K, respectively. 

Let x* be a spectrally sparse signal expressed as a sum of k complex exponentials as follows: 

k k 


= Z C I e 


irf i l _ 
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j =i 
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where /* e [0,1] represents a frequency, c* = c*je l A is its coefficient, and 0 * e [0. 2 tt] is 
its phase, J\f - {l : 0 < l < n - 1, l e N} is the set of time indices. Here, a(f*. 0 *) e 
is a frequency-atom, with the Z-th element given by a(f*,4>*)i = e l ( 27r /jj n particular, 
when phase is 0, we denote the frequency-atom simply as a(fj). We assume that the signal 
in (III. II) is observed over the time index set M c A r , \M\ = m < n, where m observations are 
chosen randomly. Our goal is to recover all the frequencies with the smallest possible number 
of observations. Estimating frequencies is not trivial because they are in continuous domain, and 
their phases and magnitudes are also unknown. 

The atomic norm of a signal x and its dual norm [51 Eq. (II.7)] are defined respectively as 
follows: 


INU = inf{E M : x = I c i a (/j)}. 

3 3 

IM[a= sup (q,x) R= sup (g,e I 0 a(/,O)) R = sup |(g,a(/))|, 
lhlU<l MO& r], /€[0,1] 

/e[0,l] 


(H-2) 


(II. 3 ) 


where (q, x)p represents the real part of the inner product x H q. Here, the superscript H is used 
for the conjugate transpose. In [5], the authors proposed the following atomic norm minimization 
to recover a spectrally sparse signal x * using randomly chosen time samples M £ Af: 

minimize ||x||^i subject to Xj = x*, jeJU. (II. 4 ) 


The dual problem of (III.4I) is 
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maximize (qM,x* M )m. subject to qjq^M = 0, ||g||^ < 1. (II.5) 

q 

The constraint ||g||^ < 1 in (III.5I) can be changed to supj e [ 0 ^ \{q,a(f))\ < 1 using (III.3I) . We label 
(q. a(f)) as the dual polynomial Q(f). Since the Slater condition is satisfied in (III.4I) . there is 
no duality gap between (III.4I) and (III.5I) lfl5ll . Moreover, the estimated spectral content comprises 
the frequencies at which the absolute value of the dual polynomial, which is derived from q 
(obtained as a solution of (111.51) 1. attains the maximum modulus of unity. We refer the reader 
to 0 for details. The off-the-grid compressed sensing approach in 0 demonstrated that with 
randomly chosen observation data, one can correctly obtain frequency information by solving the 
atomic norm minimization. However, the atomic norm minimization requires a certain minimum 
separation between frequencies for successful recovery. 

In 0, we considered frequency recovery with external prior information, and showed that if the 
frequencies are known to lie in frequency subbands, we can obtain better recovery performance 
by using frequency block prior information. The SDP formulation adopted inside each iteration 
of our new algorithms follows that detailed in 0. We summarize that SDP formulation in the 
following paragraph. 

Suppose the frequency / of the signal x lies within the frequency block B c [0,1]. Then, 
given this block prior information B, the atomic norm with block priors and its dual are stated 
respectively as follows 0 Eq. (II.9)]: 

\\x\\a,B = inf{X!M : x = fj 6 (H-6) 

3 3 

h\\*A,B= su P (?,a:>R = sup|(g,a(/))|. (II.7) 

IMU,b<i 

We formulate the atomic norm minimization with block priors 0 Eq. (III. 1)] as 

minimize ||x||^.b subject to Xj=x*-, jeM. (II.8) 

The dual problem of (III.81) is 

maximize {q M ,x M )R subject to = 0, |M|^ e < 1. (H.9) 

Here, B is a union of disjoint frequency blocks within which all the true frequencies are located, 
i.e., f* € B, B - u[. = 1 [ f Lk , frr l; \, where r is the number of disjoint block blocks, f Lk and f Hk are 
the lowest and highest frequencies of the k- th frequency block. Using the properties of positive 
trigonometric polynomials [T6i 17] and (III.7b . this dual problem can be formulated as an SDP 
0 Eq. (III. 16)]: 
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maximize ( q M ,xm)i 

9.{Ga i }E. 1 ,{G» 4 }E.i 


subject to qjq-s.M = 0, 

4 * =C kijLi j Hi (G ai ,G bi ), '■ 


(II. 10) 


G ai q 

q H 1 


> 0, * = 1 


where 5 ki - 1 if k, = 0, and 5 ki = 0 otherwise, G ai e C nxn and G bi e r( n - l ) y '( n - 1 ) are Gram 
matrices. The trace parameterization term C k j L j H (G a ,G b ) for the frequency block [f L . f H ] c 
[0,1] is set to tr[@ fc G a ] +tr[(di@ fc _i + d 0 @ k + df& k + 1 ) • G b ], where Q k is the Toeplitz matrix 
that has ones on the k-\h diagonal and zeros elsewhere, d 0 = -^p-, d\ = where 

a = tan(27r/ L /2), and 0 = tan(27r/^/2) when [f Ll f H ] <= [0,0.5], and a = tan(27r(/ L - l)/2), 
and 0 = tan(27r(/^ - l)/2) when [f L , f H ] c (0.5,1]. This SDP approach 0 was expanded for 
more general cases in [|T4ll . 

Although the atomic norm minimization that exploits external prior information can improve 
signal recovery performance, in practice, one may not always have direct access to prior in¬ 
formation. This leads to the question if we can improve the frequency recovery performance 
without any external prior information. We describe new algorithms to address this issue in the 
following section. 


III. Block iterative (re)weighted Atomic Norm Minimization Algorithms 

We propose three iterative algorithms to enhance frequency recovery performance in the 
absence of external prior information. In our algorithms, we use estimated frequency support 
information from previous iterations as block prior for subsequent iterations. 


A. Block iterative weighted Atomic Norm Minimization 

We first introduce a conceptual algorithm named Block iterative weighted Atomic Norm 
Minimization (BANM). BANM solves SDP of (III. 101) repeatedly, using block priors obtained 
from the previous iteration. In each iteration, BANM estimates the frequency locations, and 
then, around the estimated frequencies, BANM forms blocks which very likely contain the true 
frequencies. With the block priors so obtained, BANM enhances frequency recovery via solving 
(III. 101) in the next iteration, using the new block information. 

BANM initially sets the iteration number t - 0, frequency block B - [0,1], / e B. and then 
solves (III. 101) . Suppose the solution of (III. 101) gives r estimated frequencies i = 1,..., r, where 
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the superscript (£) is used to represent the iteration number. BANM chooses the l frequencies 
f n , f l2 , ..., f tl with the largest coefficients in amplitude among them, where l is a certain integer 
number. BANM then forms a union frequency block B with l frequency subbands around the 
estimated frequencies f)'\ j = 1as 


B=UL fif-Tj? 

3 = 1 


+ r], 


(III.l) 


for some small real number r>0, re [0,1] that determines the size of the subband. BANM uses 
the union frequency block B as block prior and solves (III.l01) again with updated parameters. The 
algorithm continues solving (III. 101) and updating (IIII. lb in each iteration until either a maximum 
number of iterations is reached or the solution of (III. 101) converges. 


B. Block iterative reweighted £\ and Atomic Norm Minimization Mixture 

BANM requires solving SDP in each iteration to estimate the location of frequencies, but 
solving SDP repeatedly causes long execution time. Thus, we propose using low-complexity 
algorithm to obtain prior information on frequency locations. We then use the aforementioned 
SDP (III. 101) only in the last iteration for accurately determining the frequency locations. This 
concept is the key to design of our algorithm - Block iterative reweighted £\ and Atomic Norm 
Minimization Mixture (or simply, BANM-Mix) algorithm that can achieve super-resolution of 
frequencies with low complexity (Algorithm [I])- 

BANM-Mix first discretizes the continuous frequency domain [0,1] in uniform intervals of 
size A j. We denote the index set for these intervals as V = where p = 1/A f. The index 

set corresponds to p discrete frequency grid points f :j - (j - l)/p, 1 < j < p. We have the discrete 
Fourier matrix F e C nxp over p discrete frequency grid points whose element in the j- th column 
and l -th row is a(fj)i = e l ( 27 T B 0. 

Then, BANM-Mix iteratively solves reweighted l\ minimization over this discretized frequency 
dictionary to efficiently estimate frequency locations. Different from iterative reweighted i\ mini¬ 
mization algorithms designed for incoherent discrete dictionaries If8l-fl2l. our iterative reweighted 
i\ minimization algorithm employs novel adaptive gridding and block reweighting strategies to 
extract frequency support information from our highly correlated discretized dictionary. 

BANM-Mix initiali z es coefficients = 0, weights w < f Y ' > = 1 for i = 1.....p, and an index set 
/c (0) = {j:j = ql + 1, l = 0,1,..., (p - 1 )/q, q € Z + } = {j} j= v . q:p <= V. Let W^) = diag(w^J t) ) be a 
diagonal matrix with weights w^ t) , F K ( t ) 6 0 M \ X F <I} \ be the partial discrete Fourier matrix. 
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Fig. 1. An illustration of the adaptive gridding. The estimated frequency f 2 q+i in the first iteration is depicted by a red pole. 
The index set K, and V have solid and dotted grid points respectively. The block that contains the red pole in the middle is the 
index-wise frequency block Bl 2 q + i, where b = 10. 


In the t-th iteration, BANM-Mix solves the following weighted /j minimization problem over 
the index set rather than the larger index set V: 

minimize ||W^,z||i subject to arf = (F^z)^ leM. (HI.2) 

Z 

We then define a vector c® having = z and c^ KW = 0. 

BANM-Mix then calculates the weight w[ t+1 \ i = 1, We define the index-wise frequency 
block Eli as 

Bli = {j : i - b/2 < j < i + 6/2, jeV}, (III.3) 

for some positive integer b, which determines the block width. BANM-Mix computes the weight 
ujf +1) by considering the frequency coefficients around /j in the discretized domain as 

w i t+1) = -7777-. i = (HI- 4 ) 

ZjtBlM l + c 

where e is a small positive constant to prevent wf +1 ^ from going to infinity. We refer to our 
procedure in (IIII.4I) as block reweighting. Since the discretized dictionary under consideration 
has highly correlated columns, block reweighting can accurately reflect the likelihood of a 
true frequency existing around /j. Our numerical experiments showed that earlier reweighting 
strategies [fSHTTll. which update w, «- t-[—, could not correctly reflect the likelihood of a true 

|Cj|+€ 

frequency being at index i and resulted in worse frequency recovery performance. This is because 
the solution to (IIII.2I) will disperse the amplitude of a true frequency into the neighboring indices 
in highly correlated dictionary columns. 

After updating wf +l ^ for i - 1 BANM-Mix updates the index set !C (ul i through adaptive 
gridding. In adaptive gridding, BANM-Mix first finds indices i, 1 < i < p, with wf +1 ^ < 



















Algorithm 1 : Block iterative reweighted l\ and Atomic Norm Minimization Mixture 
(BANM-Mix) Algorithm 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 


Input: F e C nxp , x* M , Maxltr, b, e , e err 
Output: frequency /, coefficient c 

Initialize: t <- 0, «- 0, <- 1, /C^ <- {i}i=i:q: P , P «- {i}? =1 

for t = 1 to Maxltr do 

C K(*) solution of QII-21 . Cp^ct) ■*- 0 
BTi *- frequency block via OIL 3b for i = 1 
«- weight via rani for i = 1, ...,p 

if ||c (t_1> -c^^||2 < £err then 

£> «- frequency block via OIl.lt . where /; satisfying w^ t+1 ^ < (min(wI t+1 b+max(™ ( ‘ +1) ))/ 2 
/ <- / such that |Q(/)| = 1 in B after solving OI.lQt 
c *- c satisfying linear equation <unj with given / and x* M 
break 

end 

KL ( - t+1 ' > «- index set via m 

end 


(min(in( t+1 ))+max(«i( i+1 ))Y 2; where min(u>( t+1 )) and max(wF +1 )) are the minimum and maximum 
values of the elements of u>( t+1 ) respectively. We define (minO( t+1) )+max(«/ t+1 >)y 2 as 1 *. Then 
BANM-Mix updates /C( t+1 ) as 

/C (t+1) =/C ( ‘) U ( U BX X 

' {r-w^<w^\i*V} > (in 5) 

Namely, if < w^\ /C (,+| ) will include finer grid points (with separation Aj) around 

frequency (i - 1 )/p. Recall that, at the beginning, /C ( °) has only grid points with separation 
q A j . The reason is that when wf +1>> is small, very likely a true frequency exists around 
frequency (i - 1 )/p. By applying finer gridding around frequency (i - 1 )/p, one can estimate 
the frequency location more accurately in the next iteration. We call this method of applying 
different resolutions in the discretized dictionary as adaptive gridding (see Fig. [Ql 

The algorithm continues solving (IIII.2I) in each iteration until either a specified maximum 
number of iterations (Maxltr) is exhausted or the solution of (IIII.2I) converges i.e., || 2 < 

e err , for some error tolerance e err > 0. BANM-Mix then chooses the block prior set B by a union 
of the frequency blocks around frequency satisfying ujf ' V) < w ^ d 1 *. With this frequency 


block information, we use SDP (III .101) to super-resolve frequencies in the last iteration. 

























C. Block iterative reweighted i\ Minimization 

The complexity of BANM-Mix can still be high since we have to solve an SDP in the 
last iteration. To further reduce its complexity, we propose the Block iterative reweighted l\ 
Minimization (BL1M) algorithm which is the same as BANM-Mix except that BL1M does 
not solve SDP in the last iteration. Instead, BL1M uses postprocessing to estimate the final 
frequencies from the results of iterative reweighted i\ minimizations. In the last iteration, BL1M 
finds the frequency blocks BI, that satisfy «;■ , +1 ^ < u^ M 1} . If two frequency blocks BI, overlap, 
BL1M merges them into one. BL1M assumes that one frequency block contains only one true 
frequency. Suppose that one frequency block (after possible merging) has r grid frequencies 
whose corresponding coefficients are ci,...,c r . Then BL1M estimates the frequency / 
in that block as / = 

J Xi=i |c;| 


IV. Numerical Experiments 

We compare our algorithms with the standard Atomic Norm Minimization (ANM) 0, and 
the Reweighted Atomic norm Minimization (RAM) ||T3l . We use CVX llT8ll to solve convex 
programs!] In all experiments, the phases and frequencies are sampled uniformly at random in 
[0,27 t) and [0,1] respectively. The amplitudes \cj\, j = 1..... /,:, are drawn randomly from the 
distribution -y/0.5 + xj where xf represents the chi-squared distribution with 1 degree of freedom. 

We evaluate the recovery performance for the signal dimension n - 64, number of observation 
m is varied from 8 to 25, block width b - 20, e = 2 8 , and e err = 0.5 x 10 -4 . The maximum 


number of iterations (Maxltr) is set to 20 for both BANM-Mix and RAm[] Fig. [2] and [3] show the 
probability of successful recovery of the entire spectral content over 50 trials for each parameter 
setup. We consider a recovery successful if \\f* - /1| 2 < 10~ 3 . Fig. [3] clearly shows that our 
algorithm outperforms both ANM and RAM for n = 64 and k - 8. 

We assess the computational complexity of algorithms in terms of the average execution time 
for signal recovery from 10 trials. Here, we present results when n is from 120 to 470, m = [ n / 2 J, 
q = 2 4 , p - 2 14 , b = 20, e = 2 s , e err = 0.5 x 10 -4 . Fig. |4] shows that the speed of BF1M is faster 


*We conducted our numerical experiments on HP Z220 CMT with Intel Core i7-3770 dual core CPU @3.4GHz clock speed 
and 16GB DDR3 RAM. using Matlab (R2013b) on Windows 7 OS. 

2 A Maxltr value of 20 was sufficient to guarantee an empirical convergence of our iterative procedures in most of our 


experiments. 
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Fig. 2. The probability P of successful frequency recovery (n = 64). 



Fig. 3. The probability P of frequency recovery for ( n,k ) = (64,8). 



Fig. 4. The execution time as a function of signal dimension n. 


than that of ANM and RAM. This is because the latter is based on an SDP while the former 
uses only £i minimization. 
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V. Conclusion 

The BANM-Mix and BL1M show better recovery than other known iterative methods |]5l 
[T31 . In particular, BL1M has shorter execution times than these other methods. Our simulations 
empirically exhibit convergence of our iterative procedures. It would be interesting to perform 
more comprehensive theoretical analysis of convergence in the future. 
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